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ABSTRACT 

The ultimate goal of conducting an accident investigation is to prevent similar 
accidents from happening again and to make operations safer system-wide. Based 
on the findings extracted from the investigation, the “lesson learned” becomes a 
genuine part of the safety database making risk management available to safety 
analysts. The airline industry is no exception. In the US, the FAA has advocated the 
usage of the System Safety concept in enhancing safety since 2000. Yet, in today’s 
usage of System Safety, the airline industry mainly focuses on risk management, 
which is a reactive process of the System Safety discipline. In order to extend the 
merit of System Safety and to prevent accidents beforehand, a specific System 
Safety tool needs to be applied; so a model of hazard prediction can be formed. To 
do so, the authors initiated this study by reviewing 1 89 final accident reports from 
the National Transportation Safety Board (NTSB) covering FAR Part 121 
scheduled operations. The discovered accident causes (direct hazards) were 
categorized into 10 groups — Flight Operations, Ground Crew, Turbulence, 
Maintenance, Foreign Object Damage (FOD), Flight Attendant, Air Traffic Control, 
Manufacturer, Passenger, and Federal Aviation Administration. These direct 
hazards were associated with 36 root factors prepared for an error-elimination 
model using Fault Tree Analysis (FTA), a leading tool for System Safety experts. 
An FTA block-diagram model was created, followed by a probability simulation of 
accidents. Five case studies and reports were provided in order to fully demonstrate 
the usefulness of System Safety tools in promoting airline safety. 
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INTRODUCTION 

Regardless of the slow recovery of passenger volume, the air 
transportation industry is steadily regaining its customers (Woodyard, 
2004). For example, in the Asia-Pacific region, the outbreak of Severe 
Acute Respiratory Syndrome (SARS) between 2002 and 2003 had 
discouraged passengers from traveling with airlines and substantially 
consumed airline profits. Asian passengers are now gradually rebuilding 
their confidence in air transportation because of the relief of possible 
pathological contagions (Dennis, 2003; FAA, 2004; Lu, 2003). In the 
United States (U.S.), after the disastrous September 11, 2001 (9/11) terrorist 
attacks resulting in a massive economic loss (Archibold, 2001; Eisenberg, 
2001; Kluger, 2001), public confidence in air transportation is recovering 
due to the government’s implementation of advanced technologies and 
necessary means to ensure aviation safety and airport security (Loy, 2003 
July). 

Historically, the U.S. Federal Aviation Administration (FAA) is 
responsible for fostering and encouraging civil air commerce and 
simultaneously auditing aviation safety (Adamski & Doyle, 1999; Rollo, 
2000; Wells, 1999). However, the FAA’s “dual-mandate” responsibility has 
resulted in criticism in terms of the lack of a sufficient ability to accomplish 
safety surveillance (Carlisle, 2001; Carmody, 2001; Donnelly, 2001; Filler, 
2001; Nader & Smith, 1994; Stout, 1999). Not surprisingly after 9/11, the 
FAA’s workload was immediately increased due to the urgent response to 
war on terror. In order not to overburden the FAA, the Transportation 
Security Administration, initially a new branch of the Department of 
Transportation and now attached to the Department of Homeland Security, 
was specifically created to take charge of the overall transportation safety. 
However, despite a tightened airport security, aircraft accidents that 
endanger aviation passengers still occur periodically (e.g., the crash of 
American Airlines Flight 587 in New York on November 11, 2003 and US 
Airways Flight 5481 in Charlotte, NC, on January 8, 2003). Accidents 
indicate a continuing demand to improve safety; but at the same time, most 
airlines operate with a “red-ink” balance sheet (Lu, 2003). In fact, the 
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airline industry is faced with a critical challenge: improving safety in an 
expense-reducing environment. In this situation, a practical model that 
assists safety managers in promptly identifying safety deficiencies would be 
very helpful. 


LITERATURE REVIEW 

Although the airline industry is extremely safe, finding a better way to 
continuously audit and promote aviation safety is a perpetual duty for all 
safety enthusiasts. During the past decade, several leading media reports — 
the Wall Street Journal (Dahl & Miller, 1996; Goetz, 1998) and USA Today 
(Stroller, 2000) — have tried to rank airline safety by solely focusing on a 
single element: the accident rate. In addition to the reports from Dahl and 
Miller, Goetz, and Stroller, Bowen and Lu (2000) advocated the importance 
of measuring airline safety performance and suggested a more 
comprehensive tool. As advocated by Bowen and Lu in 2000, a more real- 
time risk-audit model available for airline managers and government 
agencies could promptly help remedy potential threats to safety. In 2001, 
Bowen and Lu initiated a new safety measuring mechanism — the Aviation 
Safety Rating. This study compared 10 major airlines’ safety performance 
based on four essential categories — Enforcement Action, Accident Rate, 
Management Performance, and Financial Health — with 17 selected safety 
factors (Bowen & Lu, 2001). By applying Analytic Hierarchy Process as 
well as the national Airline Quality Rating, a relative comparison of safety 
performance among 10 US-based airlines was generated. The ASR provided 
a reference table of the airline overall safety that was available for the flying 
public and government agencies. In order to help airline managers prioritize 
the accident factors for effective safety training, Bowen and Lu (2004) 
conducted a follow-up study focusing on the criticality of selected risk 
factors affecting overall airline safety. They reported “the level of 
importance” pertaining to the selected safety factors using a new 
terminology, namely performance sensitivity (Sp). They defined Sp as: the 
percentage change of overall safety score due to the percentage change of a 
specific safety factor. Based on Sp calculation, a list of prioritized factors 
impacting safety performance was created. The result showed that fatality 
rate, average fleet age, and accident rate were the three most critical factors 
affecting an airline’s overall safety performance (Bowen & Lu, 2004a). 

Although the prior studies have proposed tools for measuring airline 
safety performance, they all had one thing in common: they did not discover 
the genuine cause of accidents. Further research is required so as to reveal 
the causality between root factors, causes, and accidents. This situation 
opens a window for further research. With the discovery of root factors 
leading to causes of accidents, a model that targets on accident prevention 
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and safety training could be formulated. In this study, the System Safety 
techniques were applied in an attempt to fill this knowledge gap. 

The System Safety concept 

System Safety was conceptualized by the U.S. aerospace industry in the 
late 1940s (Vincoli, 1993). Traditionally, System Safety experts in 
aerospace engineering applied systemic analysis to identify operational 
hazards and subsequently provide countermeasures before a mishap in order 
to eliminate potential risks or hazards (Malasky, 1982; Roland, & Moriarty, 
1990). System Safety is defined by Military Standard 882B as “the 
application of engineering and management principles, criteria, and 
techniques to optimize safety within the constraints of operational 
effectiveness, time, and cost throughout all phases of the system life cycle” 
(Layton, 1989, p.l). It is widely known that using System Safety concepts is 
an effective approach to reduce risk by identifying potential hazards, 
providing countermeasures, and assessing the outcome in relation to an 
operational system (Malasky, 1982; Roland, & Moriarty, 1990; Vincoli, 
1993). As noted by Vincoli, a countermeasure could be in the format of 
system re-modification, warning device, safety training, or regulatory 
change; and the application of a specific countermeasure is based on the 
result of cost-effect analysis. 

Risk matrix and risk chart 

System Safety is a doctrine used to minimize risk, optimize safety, and 
maximize system’s expected function (Layton, 1989; Malasky, 1982, 
Vincoli, 1993) by using a “risk matrix” (see Appendix A). In the “Risk 
Matrix”, risk is defined as the “likelihood or possibility of hazard 
consequences in terms of severity and probability” (Vincoli, 1993, p.10). To 
further explain this concept, if either the probability (the likelihood of a 
condition or a set of conditions that exist in a given environment) or 
severity (the description of hazard level based on real or perceived potential 
for causing harm, injury, or damage) or both can be minimized, the risk (R) 
of an accident will be minimized consequently. Thus, when the reduction of 
a potential risk (R) becomes urgent, the multiplication of probability (P) and 
severity (S) (i.e., Risk = Probability x Severity) can be flexibly used to 
achieve the determined safety goal (Malasky, 1982; Roland, & Moriarty, 
1990; Vincoli, 1993). To do so, a Risk Chart (see Appendix B) should be 
designed to better interpret the meaning of the original risk matrix in the 
hope of shifting the line of R 3 to R 2 or even Rj (i.e., either 
Probability/Frequency is reduced from “A: Frequent” toward “E: 
Impossible” or Severity is compressed from “I: Catastrophic” toward “VI: 
Negligible”). 
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The application of System Safety concept 

There are very few studies using System Safety in promoting aviation 
safety regardless of the common application in the fields of aerospace 
engineering, product manufacturing and design, environmental hygiene, and 
medicine. 

In the medical safety field, Robert L. Helreich (2000) advocated the 
application of the System Safety error management concept in medical 
practice. In his study, he first determined the origin of System Safety 
stemming from aerospace engineering and then the usefulness of data 
management pertained to hazard reduction. To accomplish hazard 
reduction, a well-managed database was the key to prevent medical 
malpractices based on the statistical predication of the likelihood of a 
failure. Yet, solely addressing the quantitative forecast, Helrireich’s study 
did not provide any workable models or procedures that the industry could 
adopt and implement. In fact, Helreich’s work was not the only application 
of System Safety techniques in medical industry. Manon Croheecke and his 
research associates (1999) and William Hyman (2002) utilized the leading 
tool of System Safety, the well-known FTA, in evaluating potential hazards 
associated with new innovated medical devices before moving toward the 
production phase within the device’s life cycle. 

In aviation safety, the military launched System Safety techniques to 
improve pilot training procedures. According to Diehl’s (1991) cross- 
referenced analysis of 208 military accidents, the top three pilot errors 
leading to mishaps were decision making, mission analysis, and situational 
awareness. Human error was found to be the major cause of aircraft 
accidents in the U.S. Air Force (Diehl, 1991). He discovered that the 
breakdown of cockpit communication/team performance, known as crew 
coordination, had directly constituted military aircraft mishaps. As a result, 
a mandatory crew and cockpit resource management (CRM) training, 
developed by National Aeronautics and Space Administration (NASA), for 
military aircrews was immediately put in place. Diehl’s study also used 
System Safety techniques to suggest a modification of the cockpit layout of 
the Cessna Citation used by U.S. Air Force officers. He conducted a 
hazardous and ergonomic analysis and suggested that the cockpit control 
panel should be redesigned in order to eliminate possible confusions 
between pilots and their working environment. His study linked System 
Safety analysis, accident investigation, and hazard identification, to human 
factor and CRM training. He subsequently recommended the development 
of a user-friendly cockpit for military pilots. 

A recent study by Thom and Clariett (2004) was published in 
Collegiate Aviation Review focusing on the applicability of job safety and 
task analysis, another essential tool of System Safety. In their study, a basic 
concern of System Safety analysis, namely job safety analysis, was closely 
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interpreted and the layout of human-machine interface was emphasized. 
Using the Risk Homeostasis Theory of human behavior, their study helped 
identify potential hazards surrounding the hangar, factory, or student 
workshop both internally and externally (Thom & Clariett, 2004). This 
study was of great interest to the aviation community. This study introduced 
aviation educators to the heart of System Safety techniques (job safety, 
environmental factors, failure modes, human error, and hazardous 
categories) and developed significant interest in it within the aviation 
community. 

The previous studies showed the importance and applicability of 
aerospace engineering’s System Safety techniques in promoting military 
flight safety, reducing medical service fault and malpractice, enhancing 
cockpit design, and identifying workplace hazards. Although System Safety 
has been recognized by various industries in upgrading safety or reliability, 
only a small portion of the aviation research community have utilized 
specific System Safety tools to promote airline safety. 

The FAA ’s System Safety efforts 

The Office of System Safety is the leading player in the FAA’s work on 
aviation System Safety research. It was in 2000 that the FAA Office of 
System Safety first introduced System Safety concept to the aviation 
industry and initiated risk management workshops for its own staffers in 
Hampton, Virginia as a compliance activity after the FAA Order 8040-4 
was published (FAA, 1998). The FAA Order 8040-4 required the Office of 
System Safety to incorporate a risk management process for all high- 
consequence decisions (FAA, 1998, p.l) and to provide a handbook/manual 
of System Risk Management and to recommend “tools” of System Safety to 
all US-based airlines. In addition, an annual System Safety conference and 
workshop available for airline managers has become routine since 2000. 
The research efforts from the FAA, project contractors and other sources 
were discussed and ideas were exchanged during each conference or 
workshop. Despite the handbook of System Safety containing essential 
System Safety theories, the current System Safety publications are limited 
to engineering design; navigation system; weather and turbulence forecast; 
global positioning systems; runway incursion; consumer safety guidelines; 
and aiiport operational procedures. On the other hand, the usage of System 
Safety has been closely tied to data collection and risk management on a 
voluntary basis in the airline industry. Examples of such data collection and 
management include the Air Transportation Oversight System (ATOS), 
FAA Safety Reporting System and Database (SRSD), NASA Aviation 
Safety Reporting System (ASRS), Flight Operational Quality Assurance 
(FOQA), Air Carrier Operations System Model (ACOSM), and American 
and Delta Airlines’ Aviation Safety Action Program (ASAP) (see Appendix 
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C). It is obvious that most current studies from the airline industry have 
been limited to a basic introduction of System Safety management, data 
collection for risk analysis and trend study such as SRSD, ASAP, ASRS, or 
FOQA. Appling System Safety “tools” such as Fault Tree Analysis (FTA) 
to identify and prioritize hazardous precedents upstream, determine 
countermeasures, reduce hazardous probability or severity, and prevent 
accidents upfront throughout the life cycle of flight operation would provide 
another meaningful mechanism to the aviation community. It would also 
extend the scope of the usage of System Safety. In this paper, one of the 
essential System Safety tools, namely FTA, was adopted for the required 
calculation of hazardous probability and future simulations purpose. 

Fault tree analysis 

FTA is used to examine an extremely complex system involving 
various targets such as skills, quality, equipment, facility, operators, 
finance, management, reputation, or property within the domain of 
operation (Malasky, 1982). 

“By placing each contributing factor in its respective location 
on the tree, the investigator can accurately identify where any 
breakdowns in a system occurred, what relationship exists 
between the events, and what interface occurred” (Vincoli, 

1993, p.135) 

FTA uses an inductive approach in conjunction with Boolean logic and 
failure probability that connects a series of events leading to the top-event 
(Roland & Moriarty, 1990; Vincoli, 1993) (see Appendix D). To 
accomplish a holistic view of an aviation system facing critical hazards, 
FTA tracks upstream and identifies causal factors that may lead to an 
accident or system failure (Brown, 1976). In addition, FTA will help 
researchers build an advisory foundation (recommendation-basis) for 
developing a better accident prevention program from the bottom-up 
(Brown, 1976; Malasky, 1982). The basic procedure of conducting FTA is 
suggested as follows: 1) identifying the top-event, 2) finding all 

contributory events from top-down, and 3) creating a full “fault tree” for 
analysis and recommendation (Roland & Moriarty, 1990; Vincoli, 1993). 
Because FTA may encompass hundreds of root factors underpinning 
accident causes, this study introduced a mini-FTA model that is sufficient to 
describe its purpose of accident-prevention and safety training (Vincoli, 
1993). 
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Research focus 

In order to fulfill knowledge gap and further apply System Safety in 
promoting airlines’ operational safety, the implementation of this study was 
designed with the following four stages: 1) identifying the direct hazards 
leading to airline accidents, 2) discovering critical safety factors 
constituting the causes leading to an accident, 3) creating an accident 
prevention model using FTA for risk simulation, and 4) providing case 
studies and reports showing the applicability of FTA in commercial 
aviation safety by recommending training emphasis. 

RESEARCH TECHNIQUES 

Document review 

Accident reports (between 1999 January and 2004 May) were retrieved 
from the U.S. NTSB Accident Docket Databases focusing on FAA FAR Part 
121 scheduled U.S. air carrier services. Accident reports were limited to final 
reports meaning the accident investigation had been completed before the 
day of data retrieval and analysis of this study. 

Data coding 

Data coding is a systematic procedure for synthesizing the significant 
meanings of texts by references and comparisons across different records 
and coders (Maxwell, 1998; Miles & ]man, 1994). Data coding is a standard 
practice for a qualitative study (Gough & Scott, 2000). Based on the 
aforementioned analytical highlights of data coding, this study coded 
accident reports based on eight (8) main components: (a) name of air carrier, 
(b) date of accident, (c) aircraft type, (d) number of fatalities, (e) number of 
injuries (both serious and minor), (f) aircraft and property damage, (g) cause 
or causes of an accident, and (h) factor or factors of an accident cause or 
causes. 

Reliability and validity 

The reliability of this project rests in the category of research 
consistency. This consistency involved operational processes of Delphi 
techniques (re-identification) and the conformability of results (Creswell, 
1998; Maxwell, 1994). This study used cross-references skill of qualitative 
data coding (QDA) double-checking two codebooks obtained from different 
analytical time and researchers (August 10 and October 1). The obtained 
reliability rate was 90.9% (ten out of eleven causes were concurred where 
the code of “Weather” was not identified by one of the researchers initially). 
After a third round of data review, the cause labeled as “Weather” was 
collectively updated and placed into the cause labeled “Turbulence.” This 
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agreement was done after the initial reliability rate (90.9%) was achieved. 
About validity, the governmental information databases help researchers 
secure data validity of a qualitative research based on the value of 
verification, trustworthiness, and authenticity (Creswell, 1998). With this in 
mind, the NTSB accident reports satisfy the validity criteria of good 
qualitative research (Berg & Latin, 1994; Creswell, 1998; Lincoln & Cuba, 
1985). 


FINDINGS 

The time-period of data retrieval and analysis was between June 18 and 
December 11, 2004. There were a total of 189 final accident reports 
available on the NTSB Aviation Accident Database dated between January 
1, 1999 and May 31, 2004. The finding sections were reported as follows: 
1) The causes of airline accidents, 2) The contributing factors of accident 
causes, 3) FTA model and probability simulation, and 4) Case studies and 
FT A reports. 

The direct causes of airline accidents 

The direct causes leading to FAR Part 121 airline accidents between 
January 1999 and May 2004 were ranked and categorized as follows (see 
Table 1): 


Table 1. The Direct Causes of FAR Part 121 Airline Accidents 


Rank 

Accident Cause* 

Number of Cases 

% of Cases 

1 

Flight Operations 

46 

24.34% 

2 

Ground Crew 

43 

22.75% 

3 

Turbulence 

40 

21.16% 

4 

Maintenance 

25 

13.23% 

5 

Foreign Object Damage (FOD) 

15 

7.99% 

6 

Flight Attendant 

8 

4.23% 

7 

Air Traffic Control (ATC) 

4 

2.12% 

8 

Manufacturer 

4 

2.12% 

9 

Passenger 

3 

1.59% 

10 

FAA 

1 

0.53% 


* Please see Appendix E for the definition of each accident cause after data coding 


The accident cause due to Flight Operations error resulted in 46 
accidents (24.34%), which was the most critical individual cause of the Part 
121 accidents. There were 43 accidents as a result of Ground Crew error 
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followed by Turbulence (40 cases), Maintenance (25 cases), FOD (15 
cases), Flight Attendant (8 cases), ATC (4 cases), Manufacturer (4 cases), 
Passenger (3 cases), and the FAA (1 case). Although Flight Operations error 
was the most significant cause (24.34%), the dyad of Ground Crew and 
Maintenance (non-flight) error had resulted in 68 accidents (35.98% of the 
overall mishaps). 

The contributing factors of accident causes 

The factors leading to Flight Operations error were: 1) loss of 
situational awareness, 2) misjudgment (ground clearance), 3) weather 
(contaminated, snowy, or icy runway), 4) ineffective communication, 5) 
operational deficiency (supervision, misjudgment, preflight inspection), or 
lack of training (heavy landing, go-around procedure, unfamiliar with 
regulations, and decision-making), 6) non-compliance with standard 
operational procedures, 7) over-reaction (evasive maneuvers, abrupt 
reaction to Traffic Collision Avoid System warning), 8) physical fatigue, 9) 
weather and airport information ignorance (weather briefing, turbulence 
report. Notice to Airmen, Minimum Equipment List, outdated Runway 
Visual Range). 

The factors leading to Ground Crew error were: 1) poor situational 
awareness (clearance, airstair/jet bridge/vehicle operations), 2) ineffective 
communication (tug/truck/beltloader driver-pilots-wing walkers), 3) lack of 
supervision/quality assurance, 4) ramp agents’ ignorance of safety criteria, 
5) physical fatigue, and 6) personal health and medication. 

Most accidents due to Turbulence resulted in flight attendant injuries. 
The factors that led to injuries or fatalities resulting in the cause of 
turbulence were: 1) lack of weather awareness (pilots or dispatchers’ poor 
discipline pertaining to weather evaluation), 2) inadequate training of cabin 
crews when encountering turbulence (inaccurate cabin reaction procedures, 
ineffective crew communication, delayed public announcement), and 3) 
passengers’ inability of cooperating with cabin crews during emergency 
situation. 

The factors that led to Maintenance error (equipment contamination, 
corrosion, engine failure, etc.) were: 1) the lack of quality assurance and 
supervision on performance, 2) non-compliance of standard maintenance 
procedures (SMPs), 3) incorrect data from the FAA, 4) lack of training and 
knowledge, 5) rushed service, and 6) operational ignorance. 

The factors that led to FOD cases were: bird/geese strikes and collision 
with deer. The FOD frequently occurred during: 1) take-off and landing 
phase and 2) night flights around remote non-hub airports. The factors 
leading to the cause of Flight Attendant’ s mistakes were: 1) unfamiliarity 
with safety procedures during evacuation, 2) poor communication (between 
pilot, flight attendants, or ramp/gate agents), and 3) inadequate training with 
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abnormal emergency conditions. The factors that led to the cause of ATC 
error were: 1) improper ATC service (the result was pilot’s abrupt 
maneuver) and 2) a failure to provide adequate in-flight separation. 

The factors contributing Manufacturers’ error were: 1) inadequate 
manual information (e.g., gearbox maintenance manual), and 2) improper 
material and imperfect design. The factors that led to the cause of 
Passengers and their injuries were: 1) passengers’ non-compliance with 
regulations during emergency situation, and 2) unruly passengers and 
behaviors. The one factor leading to the cause of FAA was the FAA’s 
improper issuance of airworthiness certificate and Airworthiness Directives 
for specific parts. 

FTA model and probability simulation 

The findings revealed that there were 10 main causes, along with 36 
associated root factors, which led to airline accidents during this time 
period. A mini-FTA block diagram showed in Appendix F presents an 
inductive relationship among accidents (top level event), the accident causes 
(second level events) and the causes’ root factors (the lowest level events) 
(see Appendix F). Each accident cause contained from one to nine 
contributory root factors. Based on the Boolean logics, “AND” and “OR” 
gates, researchers are able to examine the whole system from the bottom to 
the top level. These root factors (the lowest level events) included 
inadequate flight performance, fatigue, poor quality assurance, carelessness, 
air-rage, lack of situation awareness, non-compliance with SOPs, 
miscommunication, etc. The mini-FTA model in Appendix F also 
demonstrates an individual root factor could create a category of accident 
cause (second level event) that eventually leads to an accident (top level 
event). 

To address the criticality of the 36 discovered root factors that led to 
the accidents, simulating accident probability of the top-event would help 
explain the significance of the FTA model and predict the likelihood of the 
top level event. For instance, using the study of Bowen and Lu’s assessment 
of major airlines’ safety performance in 2001 and 2004, the probability of 
pilot fatigue (a root factor) leading to an accident was about 1.7x10 “ 5 (1.7 
cases per one hundred thousand flights) (Bowen and Lu, 2004). Because 
there could be hundreds of different factors associated with one accident 
cause, the probability for an accident cause to exist would be (1.7x10 _5 ) x 
100, which is 1.7xl0" 3 (see Appendix G). And since any of the ten accident 
causes (an “OR” gate logic in this study) could lead to the top-event, the 
probability for an accident to occur could be (1.7xl0 3 ) x 10, which is 
1.7xl0' 2 meaning 1.7 accidents for every 100 flights. This high probability 
of an accident should have drawn the attention of the aviation community. 
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Reversely, based on the same FTA model presented in Appendix G, if 
airlines can reduce the accident probability of each root factor to 1.7x10 “ 7 
instead of 1.7x10 “ 5 (as a result of imposing safety trainings, new safety 
guidelines, effective flight training, or upgraded navigation technologies), 
the ultimate accident probability of the top level event becomes 1.7x1 O' 4 
meaning 1.7 mishaps for every 10,000 flights. This simulation of accident 
probability shows that it is extremely critical for the airlines to mitigate 
potential hazards from the bottom level as early as possible. If the 
probability of each root factor (the lowest level of the fault tree) could be 
compressed or even eliminated, the probability of accident causes (the 
second level of the fault tree) resulting from a combination of various root 
factors would be dramatically reduced. Eventually, the probability for the top 
level event (i.e., an accident) to occur could be minimized. 

Case studies and FTA reports 

The main purpose of conducting FTA in aviation safety is to identify 
potential hazards, provide recommendations and reports, and to prevent 
similar accidents from happening again. In order to further strengthen the 
applicability of the FTA accident model, case studies are provided. All 
cases were retrieved from the NTSB Accident Database online either in a 
PDF version. 

Case 1. NTSB ID: LAX00LA223 

An engine forward cowling door on the number 1 engine separated 
from the engine nacelle during the take off rolling at Las Vegas 
International Airport. The separated part consequently struck the 
horizontal stabilizer attached to the vertical fin. The pilot described 
that aircraft vibrated on runway during the take off rolling. The 
aircraft was under an RON (Remain Over Night) check due to the 
complexity of maintenance. The technicians opened the engine 
cowling door for the needed RON check at night but failed to ensure 
the proper hand-over procedure with the day-shift team the next 
morning. In addition to the required follow-up in relation to engine 
inspection, the day-shift team was assigned with other inspection 
tasks as well (NTSB, 2001, August 21) 

The cause and root factor of this accident was mechanic’s failure to 
refasten the cowling door prior to signing off the aircraft back to service. 
Providing countermeasures should focus on retraining communication skills 
and quality assurance and re-emphasizing team work capability based on 
the recommendations of AC- 120-5 ID and maintenance resource 
management (MRM). 
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Case 2. NTSB ID: NYC02LA013 

Before the landing, the captain briefed a “no go-around” for a night 
visual approach even though the approach was not stabilized. The 
airspeed was decreasing to near the speed of stall. After touch 
down, the aircraft maneuvered at a nose-high pitch attitude and 
struck the runway on the aft fuselage. The first officer did make an 
initial callout about the stall airspeed but the captain did not 
respond. During the post-accident interview, the captain reported 
that she decided to land without initiating go-around because there 
was no traffic on the runway at night. The first officer did not 
challenge the captain even though the decision was wrong. The 
captain described that the first officer was very quiet; yet the first 
officer complained that the captain was self-defensive and did not 
like any criticisms (NTSB, 2003a) 

The cause of this accident was the captain’s failure to maintain airspeed 
resulting in both a stall and a hard landing. The factors involved were the 
failure of both pilots to comply with the company’s CRM guidelines, flight 
manual procedures, and the captain’s improper approach briefing. 

Providing countermeasures should focus on: (a) recurrent CRM 
training, (b) pilot’s flight procedure retraining, and (c) flight operation 
proficiency and training guidelines should come from AC-120-5 ID, 
Preflight SOPs, and airline’s simulator training procedures. 

Case 3. NTSB ID: DCA99MA060 

A McDonnell Douglas DC-9-82 (MD-82) crashed after it overran the 
end of runway 4R during landing . . . After departing the end of the 
runway, the airplane failed to maintain vertical airspeed and struck 
several tubes extending outward from the left edge of the instalment 
landing system (ILS) localizer array... The airplane was destroyed by 
impact forces and a post-crash fire (NTSB, 2003b, p. 169-170). 

The cause and root factors of this accident were “The flight crew’s 
failure to discontinue the approach’’ and their failure to ensure the spoilers’ 
extension for landing due to (a) flight crew’s fatigue and stress, (b) 
situational awareness of airport weather, and (c) incorrect operation of using 
reverse thrust after landing. Providing countermeasures should focus on 
conducting recurrent CRM trainings for pilots and retraining pre and post 
landing procedures based on the recommendations of AC- 120-5 ID and 
SOPS of flight operations. 
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Case 4. NTSB ID: DCA03MA022 

A Raytheon (Beechcraft) 1900D crashed shortly after takeoff from 
runway 18R at Charlotte-Douglas International Airport due to the 
airplane’s loss of pitch control during take-off. The 2 flight 
crewmembers and 19 passengers aboard the airplane were killed, 1 
person on the ground received minor injuries (NTSB, 2004a, p. 13) 

The cause and root factors of this accident was the loss of pitch control 
resulted from an incorrect rigging of the elevator system compounded by 
the airplane’s aft center of gravity, which was substantially out of limit. 
Additional contributing factors to the cause of incorrect rigging were: (a) 
lack of oversight of the maintenance station by the airline and the FAA; (b) 
improper maintenance procedures and documentation; (c) erroneous weight 
and balance calculation; (d) ineffective manufacturer’s onsite quality 
assurance; and (e) the FAA’s outdated weight and balance assumptions. 

Providing countermeasures should focus on: (a) revising the FAA’s 
weight-and-balance reference data, (b) imposing recurrent trainings for 
quality assurance (QA) inspectors both for airline and manufacturer, (c) 
providing aircraft technician’s job compliance training, and (d) ensuring 
preflight SOPs based on the FAA’s formed rulemaking procedures and 
inspection handbooks, maintenance trouble-shooting SOPs, preflight SOPs, 
maintenance resource management (MRM) guidelines, and AC-120-51D 
recommendations. 

Case 5. NTSB ID: NYC03FA039 

A Boeing 757 was struck by a taxing Airbus, while parking at the 
gate with passengers aboard. Maintenance technicians were taxing 
the Airbus. The maintenance technicians testified that both parking 
brakes were activated while waiting for ground crews to arrive for the 
follow-up procedures. He released the parking brake after the ground 
crews arrived and took over the residual operation. The technicians 
slightly increased the throttles because the aircraft did not move after 
parking breaks were released. The airplane struck the jet way despite 
the engine throttles were repositioned to idle speed (NTSB, 2004b) 

The cause and root factors of this accident are the aircraft technician’s 
lack of training in terms of aircraft system, maintenance procedures, and 
ground safety guidelines. Providing countermeasures should focus on: (a) 
imposing a recurrent training of maintenance standard operation procedures 
(SOPs), (b) aircraft system training, and (c) ground operation safety training 
based on the maintenance resource management (MRM) guidelines, AC- 
120-5 ID recommendations, and manufacturer’s system handbooks or 
maintenance manuals. 
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CONCLUSION 

This study discovered the 10 direct causes leading to accidents and 
36 root factors behind accident causes. By using FTA, aviation safety 
practitioners can design a more efficient and effective safety training aiming 
to detect risk factors, provide countermeasures, and reduce the associated 
hazardous probability and severity. This study is concluded as follows: 

1. Implementing System Safety techniques is feasible. In this study, 
the ultimate goal of conducting System Safety analysis using FTA is to 
prevent future accidents by identifying potential hazards and providing 
countermeasures and recommendations. Although many studies had been 
accomplished measuring the overall safety performance (Bowen & Lu, 
2001 & 2004a; Dahl & Miller, 1996; Goetz, 1998; Stroller, 2000), they did 
not provide a good model for safety practitioners to promptly and 
effectively identify accident causes and their root factors. Without 
identifying specific root factors and accident causes leading to mishaps, 
solely measuring safety performance could be of limited value and result in 
aimless and ineffective safety training. In fact, System Safety experts 
advocate four fundamental levels of safety precedence regarding hazard 
ramification. They are reengineering; redundant system design; warning 
signals and devices; and safety training and education. The most 
inexpensive safety precedence is safety training and education (Vincoli, 
1993). This is an important feature for today’s airline businesses suffering 
from financial hardships and simultaneously concerned with offering the 
highest degree of care in terms of passenger’s safety. 

2. Fault Tree Analysis (FTA) is plausible. It is important to 
understand FTA because it helps safety enthusiasts (government or airlines) 
to effectively and promptly isolate accident postulates and to implement 
strategic safety prevention programs from the bottom-up. Based on the FTA 
block-diagram in this study, any of the root factors on the bottom level can 
form a cut-set, that is, a chain-of-events that can result in an accident or a 
system failure, breaking down the entire system. Hence, compressing or 
eliminating the failure probability of root factors from the lowest level of 
“the tree” should be regarded as the training priority. 

3. Human Factors training is critical to pilots. Regardless of the 
accident cause of turbulence and FOD, “pilot error” was the primary factor 
leading to airline accidents in this study. Krause (1996) and Orlady (1999) 
stated that Human Factors is a very powerful training tool for pursuing an 
error- free and safety-laden airline operation. Since 1990, the FAA has 
regulated CRM training for flight crews (based on NASA’s Human Factors 
research in the early 1970s). This can be found in Federal Aviation 
Regulation (FAR) Part 121 Subpart N for major air carriers and for Part 135 
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regional commuters under SFAR 71 (Aviation Supplies & Academics 
[ASA], 2001). 

4. Non-flight activities are equally hazardous as flight activities. 
According to the findings of this study, non-flight error constituted more 
mishaps (68 cases) than flight operation (46 cases). In fact, the aviation 
safety net consists of flight crews, maintenance personnel, air traffic 
controllers, airplane dispatchers, flight attendants, ramp agents, airport 
security, and all related professionals. Aviation personnel should work 
closely together because a single flawed portion of the safety net could 
result in an unrecoverable safety breakdown and, thereby, human injuries, 
fatalities, or substantial financial loss. By virtue of the Swiss-cheese safety 
model, aviation accidents happen when unsafe acts or operations are present 
and line up simultaneously (Reason, 1990; Wood, 1997). With this in mind, 
in order to strengthen the aviation safety net based on mini-FTA model, it 
may be reasonable for the aviation community to support a mandatory 
Fluman Factors or MRM training for ground and maintenance personnel. 

COMMENTS 

Although the potential cost is always a big concern regarding an 
accident prevention program (Del Valle, 1997; Duke, 1999; Finder, 1999; 
Flahn, 1997; Morris, 2001; Morris, Rigavan, Whitelaw, Glasser, Strobel, & 
Eltahawy, 1999; Wald, 2000), providing safety trainings to employees 
would consume the least amount of financial sources. According to System 
Safety guidelines, the prevailing methods of implementing an accident 
prevention program include system re-engineering, administrative reform, 
and work practice controls (Brown, 1976; Gloss & Wardle, 1984). If system 
re-engineering and administrative reform are too costly to adopt, work 
practice control (i.e., safety training) is the most cost-effect method to 
reduce risks and prevent potential accidents. The safety training should be 
mandatory or routine. Otherwise, the effectiveness of training would be 
lower-than-expected (Bowen & Lu, 2004b; Lu, 2003; Vincoli, 1993). 

The doctrine of System Safety is very useful in accident prevention and 
safety enhancement. Aviation safety enthusiasts could utilize System Safety 
tools like the FTA model to identify potential hazards associated with 
airline operation and to recommend needed countermeasures and trainings 
for employees. Despite the immediate goal for the aviation industry to 
regain its revenue after the 9/11, maintaining a risk-free aviation 
environment should be positioned as the top priority for airlines and our 
government. Even though the airline industry is extremely safe in the U.S., 
accidents are still a threat to the flying public because accidents will occur 
periodically and will claim lives again. From the public’s standpoint, each 
accident is a metaphor for either the government’s or the airline’s failure to 
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adequately protect its clients. This study has demonstrated that using 
System Safety tool is another viable approach to achieve the goal of zero 
accidents. 


FUTURE STUDY 

Despite free publications offered by the FAA regarding severe weather, 
in order to proactively reduce aircraft accidents resulting from turbulence 
and bird hazard/FOD, the aviation community needs to put more effort into 
meteorological, technological, and biological studies. In the future 
application of System Safety techniques, using computer software could 
dramatically help System Safety managers in different segments of the 
aviation industry simulate hazards and provide safety trainings scenarios 
promptly and accurately. With the help of computer technologies tailored for 
risk analysis, the application of FT A or other System Safety tools can be 
applied to a greater extent. 
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APPENDIX A 

RISK MATRIX, SEVERITY & PROBABILITY 

Risk Matrix* 


Frequency 

Catastrophic (I) 

Critical (II) 

Marginal (III) 

Negligible (IV) 

Frequent (A) 

1A 

2A 

3A 

4A 

Probable (B) 

IB 

2B 

3B 

4B 

Occasional (C) 

1C 

2C 

3C 

4C 

Remote (D) 

ID 

2D 

3D 

4D 

Impossible (E) 

IE 

2E 

3E 

4E 


* A “Risk” falling into this category [1A, 2 A, 3 A, 4A, IB, 2B, 1C] is “Unacceptable” 

A “Risk” falling into this category [ID, 2C, 3B, 3C, 4B] is “Undesirable” 

A “Risk” falling into this category [IE, 2D, 2E, 3D, 4C] is “Acceptable With Review” 

A “Risk” falling into this category [3E, 4D, 4E] is “Acceptable Without Review” 

The determination of “Unacceptable,” “Undesirable,” “Acceptable With Review,” or “Acceptable 
without Review” is based on a System Safety analyst’s subjective decision-making based on the 
onsite situation from case to case. 

Risk Severity (S) and Probability (P) are defined as: 

Risk Severity (S) 

Description Category Mishap Definition 

Catastrophic I Death or system loss/failure 

Critical II Severity injury, occupational illness, or system damage 

Marginal III Minor injury, occupational illness, or system damage 

Negligible IV Other 

Risk Probability (P) 

Description 
Frequent 
Probable 
Occasional 
Remote 
Impossible 


Level Mishap Definition 
A Likely to occur frequently 
B Will occur several times during the life of an item 
C Likely to occur sometimes in the life of an item 

D Unlikely, but may possibly occur in life of an item 

E So unlikely, assumed that hazard will not occur at all 


Source: DOD MIL-STD-882B System Safety Program Requirements (1984) 
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APPENDIX B 
RISK CHART 



Note. The product of Risk Probability (P) and Risk Severity (S) is equal to Potential Risk (R) 
thus in System Safety concept R = P x S. The forming of a “Risk Chart” above was converted 
from the original Risk Matrix and generates a bivariate curve for a better understand and 
interpretation. 


Lu, Wetmore and Przetak 


135 


APPENDIX C 

SYSTEM SAFETY WORKSHOPS AND CONFERENCES - 
CONTENT ANALYSIS 



2001 

2002 

2003 

2004 

System Safety Management 

X 

X 

X 

X 

Aviation System Safety Program (AvSP) 

X 

X 

X 

X 

FAA-Airlines Collaboration 

X 

X 

X 

X 

Data Collection & Risk Analysis 

X 

X 

X 

X 

System Risk Management (SRM) & Safety Culturs 


X 

X 

X 

Flight crews-centered 

X 

X 


X 

Non-flight crews-centered 

X 

X 

X 

X 

All aviation workers 

X 




Air Carrier Operations System Model (ACOSM) 

X 




Aviation Safety Action Program (ASAP) 

X 

X 


X 

Flight Operational Quality Assurance (FOQA) 

X 

X 


X 

Advanced Quality Program (AQP) 

X 




Aviation Safety Reporting System (ASRS) 

X 



X 

Continuous Analysis and Surveillance System (CA 

X 




Maintenance Resource Management (MRM) traini 

X 

X 



Human Factor CRM training 

X 

X 

X 

X 

Case-based training/Naturalistic Decision-making 

X 

X 

X 

X 

Regulations 

X 

X 

X 


Cost-benefit and Safety Investment 

X 

X 

X 

X 

Failure Mode and Effective Analysis (FMEA) 
Concept 

Failure Mode and Effective Analysis (FMEA) 
Application 


X 



Fault Tree Analysis (FTA) Concept 
Fault Tree Analysis (FTA) Application 
Risk Control Management (RCA) 


X 


X 

Hybrid Causal Modeling 



X 

X 


Note. The origin of this Content Analysis Table was statistically extracted 
from the research projects and papers presented at the FAA System Safety 
workshops and conferences between 2000 and 2004. As shown in the above 
table, most researches either focused on the advocate of using System Safety 
concepts or risk analysis covering trend study. Researchers did not apply 
tools (i.e, FTA or FMEA) to their studies for a demonstration. Especially, 
there were only two papers explained FMEA and FTA techniques over the 
past four years. Yet no further application was found. 
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APPENDIX D 

BASIC LOGICS OF FAULT TREE ANALYSIS 



Sub- Sub- Sub- Sub- Sub- Sub- 


Causes 1 Causes 2 Causes 3 Causes 4 Causes 5 Causes 6 

Pi P 2 P 3 P 4 P 5 P 6 


Note. The Sub-Causes must be preconditions of the upper level accident Cause; and Causes 
are preconditions of the Top-Event/Accident. Pi (i = 1~9) represents the risk 
probability associated with each specific “cause” or “factor.” 

Note: represents “AND” gate, while represents “OR” gate. Other logical gates could 


be used into tree analysis based on different cases, purposes or situations. 
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APPENDIX E 

TERMINOLOGY OF ACCIDENT CAUSES 


In this study, the causes leading to an accident were categorized and defined 
as the following for a better understanding of research findings: 

Flight operation : an accident was caused by cockpit crews 

Turbulence : an accident was caused by turbulence (in-flight, clear air, wake 
turbulence) 

Maintenance : an accident was caused by aircraft maintenance personnel 

Ground crew : an accident was caused by ground crews (truck driver, 
beltloader or tug operator, ramp agents, etc.) 

Foreign Object Damage (FOD) : an accident was caused by birds, animals, 
and any objects that do not belong to aircraft itself 

Flight Attendant : an accident was caused by flight attendant’s inadequate 
emergency actions 

Air Traffic Control (ATC) : an accident was caused by air traffic controller’s 
misjudgment 

Manufacturer : an accident was due to manufacturer’s design, official 
inspection manuals, etc. 

Passenger : an accident was caused by passengers themselves 

FAA : an accident was caused by FAA’s discretionary function regarding 
certificate approval, inspection, etc. 

Non-flight Error : a combination of maintenance and ground crew’s 
operational mistakes. 
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APPENDIX F 
FAULT TREE ANALYSIS 
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APPENDIX G 

SIMULATING THE PROBABILITY OF THE TOP-LEVEL EVENT 



(fi, where i = 1~100) for 
each Accident Cause 












